Networks and distributed storage allow data and storage space to be shared between devices located anywhere a connection is available. These implementations may range from a single machine offering a shared drive over a home network to an enterprise-class cloud storage array with multiple copies of data distributed throughout the world. Larger implementations may incorporate Network Attached Storage (NAS) devices, Storage Area Network (SAN) devices, and other configurations of storage elements and controllers in order to provide data and manage its flow. Improvements in distributed storage have given rise to a cycle where applications demand increasing amounts of data delivered with reduced latency, greater reliability, and greater throughput. Hand-in-hand with this trend, system administrators have taken advantage of falling storage prices to add capacity wherever possible.
One consequence of the abundance of cheap storage is the need to protect and recover increasing amounts of data. Even though storage devices have become more reliable, they are not infallible. At the system level, multiple storage devices may be grouped in a RAID array or other grouping configured to provide redundancy using parity, mirroring, or other techniques. In theory, should a device fail, the storage system can recover lost data from the remaining devices. In practice, the probability of multiple-device failures increases with each storage device added, and any data protection scheme has a limit to how many concurrent failures can be tolerated. As a result, it is still possible for a catastrophic failure to exceed the ability of the RAID array to recover.
Other techniques for data recovery leverage the distributed nature of some storage environments. For example, a storage environment may be arranged as a cluster of discrete systems (e.g., storage nodes) coupled by a network. Copies of data and/or recovery information may be distributed over the storage nodes so that data is not lost should an entire storage node fail. However, when a node fails, the amount of data transferred over the network to rebuild the node may be several times larger than the amount contained in the node. For example, rebuilding a 200 TB node may entail transferring 1.2 PB or more between nodes. Even if the rebuild process does not halt all data transactions, the network burden may severely delay those transactions still being processed.
Accordingly, a technique for distributing data and for recovering data in the event of a node failure without rebuilding a node entirely would provide numerous practical and real-world advantages. Thus, while existing techniques for data recovery have been generally adequate, the techniques described herein provide a robust data protection scheme with greater recovery options.